A Max-Sum algorithm for training discrete neural networks

A Max-Sum algorithm for training discrete neural networks

We present an efficient learning algorithm for the problem of training neural networks with discrete synapses, a well-known hard (NP-complete) discrete optimization problem. The algorithm is a variant of the so-called Max-Sum (MS) algorithm. In particular, we show how, for bounded integer weights with q distinct states and independent concave a priori distribution (e.g. l1 regularization), the ...

A Smoothing Algorithm for Training Max-min Neural Networks

In this paper, a smoothing algorithm for training max-min neural networks is proposed. Specifically, we apply a smooth function to approximate functions and use this smoothing technique twice, once to eliminate the inner operator and once to eliminate the operator. In place of actual network output by its approximation function, we use all partial derivatives of the approximation function with ...

Conjugate gradient algorithm for training neural networks

Decision Sum-Product-Max Networks

Sum-Product Networks (SPNs) were recently proposed as a new class of probabilistic graphical models that guarantee tractable inference, even on models with high-treewidth. In this paper, we propose a new extension to SPNs, called Decision Sum-Product-Max Networks (Decision-SPMNs), that makes SPNs suitable for discrete multi-stage decision problems. We present an algorithm that solves Decision-S...

A Hybrid Continuous Max-Sum Algorithm for Decentralised Coordination

In this paper we tackle the problem of coordinating multiple decentralised agents with continuous state variables. Specifically we propose a hybrid approach, which combines the maxsum algorithm with continuous non-linear optimisation methods. We show that, for problems with acyclic factor graph representations, for suitable parameter choices and sufficiently fine state space discretisations, ou...